Evaluation of the RoboCup Logistics League and Derived Criteria for Future Competitions

In the RoboCup Logistics League (RCLL), games are governed by a semi-autonomous referee box. It also records tremendous amounts of data about state changes of the game or communication with the robots. In this paper, we analyze the data of the 2014 competition by means of Key Performance Indicators (KPI). KPIs are used in industrial environments to evaluate the performance of production systems. Applying adapted KPIs to the RCLL provides interesting insights about the strategies of the robot teams. When aiming for more realistic industrial properties with a 24/7 production, where teams perform shifts (without intermediate environment reset), KPIs could be a means to score the game. This could be tried first in a simulation sub-league.